Constructive Neural Network Algorithms for Function Approximation Tasks

The generalization capability and training time of conventional neural networks depend on their architecture. In conventional neural networks, we have to define the architecture prior to training but in constructive neural network (CoNN) algorithms the network architecture is constructed during the training process. This paper presents an overview of CoNN algorithms that constructing feedforward architecture for function approximation tasks. Cascade-Correlation algorithm (CCA) and Dynamic node creation algorithm are well known and widely used for both classification and regression tasks. Cascade 2 algorithm is a variant of CCA that is found to be more suitable for regression problems and is reviewed in this paper. Our recently proposed two constructive algorithms using adaptive slope sigmoidal function are given at the end of the paper. The role of adaptive sigmoidal activation function has been verified in constructive neural networks for better generalization performance and lesser training time.